{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Novel uses of machine learning:
Predicting probabilities and trusting outputs\n", "\n", "Tom Charnock\n", "\n", "Institut d'Astrophysique de Paris\n", "


\n", "Notebook: presentations.charnock.fr/ED\n", "\n", "\n", " \n", "
\n", " \"Sorbonne\n", " \n", " \"ANR\"\n", " \n", " \"IAP\"\n", " \n", " \"CNRS\"\n", " \n", " \"Aquila\"\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Who am I?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Tom Charnock\n", "\n", "- Stastistican\n", "- Cosmologist\n", "- Machine learning safety and interpretability enthusiast\n", "- TensorFlow fan" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Cool things I've done with machine learning" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### LSTMs in astronomy (2015)\n", "Predicting supernovae type directly from lightcurves\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### DCGAN for generating galaxy images (2016)\n", "Using postage stamps from SDSS-III\n", "\n", "\n", " \n", "
\n", " \n", " \n", " \n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### Improvements using Wasserstein distance (2018)\n", "Using Horizon-AGN simulation for training\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Physically motivated mapping networks (this week)\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### WGAN-like training algorithm\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Generator maps known distribution to some physically related quantity\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Architecture chosen to pull out physical intution\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Need a critic which is able to be informative on the physical scale of relevance\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "\n", "

\n", "\n", "- Generate simulation\n", "- Pass simulation and real data through critic\n", "- Update weights of critic until converged (approx. Wasserstein)\n", "- Fix critic weights\n", "- Single update of the generator" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Neural density estimators" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Stats time\n", "\n", "Data ${\\bf d}$ described by parameters $\\boldsymbol{\\theta}$ with likelihood $\\mathcal{L}({\\bf d}|\\boldsymbol{\\theta})$\n", "\n", "Posterior of parameters $\\boldsymbol{\\theta}$ found using Bayes theorem\n", "\n", "$$\\mathcal{P}(\\boldsymbol{\\theta}|{\\bf d})=\\frac{\\mathcal{L}({\\bf d}|\\boldsymbol{\\theta})p(\\boldsymbol{\\theta})}{p({\\bf d})}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# We can use NDEs to approximate
$\\mathcal{P}(\\boldsymbol{\\theta}|{\\bf d})$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Mixture density networks\n", "_Any_ function* can be approximated with a mixture of distributions\n", "
\n", "\n", "\n", "\n", "\n", "*under certain conditions" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### We can simply fit the parameters of the distributions using regression!\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# Hey presto - we've got the posterior!" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# MDNs are not only for probabilities" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### We can learn generic functions\n", " " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### The density field really maps to a random realisation of halo counts - it's none deterministic. \n", "

\n", "\n", "\n", "### We can learn the distribution of halo counts!" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Autoregressive flows" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Improving on MDNs\n", "\n", "- With MDNs we have to assume a final distribution (mixture of Gaussians, etc.)\n", "\n", "- We also work only on the posterior distribution (or the likelihood if we work backwards)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Learning the whole probability space\n", "\n", "The joint space of data and parameters $\\boldsymbol{\\psi}=\\{{\\bf d}, \\boldsymbol{\\theta}\\}$ and networks help learn the true distribution\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Normalising flows\n", "\n", "A simple way to learn this space is by making a guess for the distribution and then squashing and squeezing it into place.\n", "\n", "This can be done using normalising flows $\\mathscr{f}:\\boldsymbol{\\psi}\\to\\boldsymbol{\\chi}$ - a series of invertible transformation of the initial distribution\n", "$$\\mathscr{f} = \\mathscr{f}_0 \\odot \\mathscr{f}_1 \\odot \\cdots \\odot \\mathscr{f}_k$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "The joint distribution (which can be complex and weird) is found using\n", "\n", "$$\\mathcal{P}(\\boldsymbol{\\psi}) = \\mathcal{P}_\\boldsymbol{\\chi}(\\mathscr{f}^{-1}(\\boldsymbol{\\psi}))\\left|\\textrm{det}\\frac{\\partial\\mathscr{f}^{-1}}{\\partial\\boldsymbol{\\psi}}\\right|$$\n", "\n", "where the initial guess $\\mathcal{P}_\\boldsymbol{\\chi}$ can be simple.\n", "\n", "### We just need to find the transformation $\\mathscr{f}$!" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Autoregressive models\n", "\n", "We can build probability distribution out of conditional probability distributions\n", "\n", "$$\\mathcal{P}(\\boldsymbol{\\psi})=\\prod_{d=1}^D\\mathcal{P}(\\psi_d|\\psi_{1:d−1})$$\n", "\n", "Each conditional probability distribution depends on all of the previous inputs from the input vector." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "### We model the conditionals with simple distributions\n", "\n", "A neural network with variable input sizes predicts MDN-like parameters\n", "- The number of inputs defines the conditionality of the distribution of the coniditional distributions" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Cosmic shear inference\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Cosmological parameter constraints\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# How can we trust our networks?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### No networks are impervious to mode collapse\n", "\n", "It would be great if we could understand how uncertain we are about our network's outputs.\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "To do so we must think statistically again!" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Stats time (again)\n", "\n", "A network is just a universal map\n", "\n", "$$\\mathscr{f}_\\boldsymbol{v}:\\mathbb{R}^{d}\\to \\mathbb{R}^{x}$$\n", "\n", "where an input ${\\bf d}$ is mapped to an output ${\\bf x}=\\mathscr{f}({\\bf d}, \\boldsymbol{v})$ where $\\mathscr{f}$ is tuned via trainable parameters, $\\boldsymbol{v}$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "The probability of getting output ${\\bf x}$ given some set of network parameters $\\boldsymbol{v}$ and input data ${\\bf d}$ is described by $\\mathcal{L}({\\bf x}|\\boldsymbol{v},{\\bf d})$.\n", "\n", "$$\\mathcal{P}(\\boldsymbol{v}|{\\bf x},{\\bf d}) = \\frac{\\mathcal{L}({\\bf x}|\\boldsymbol{v},{\\bf d})p(\\boldsymbol{v}|{\\bf d})}{p({\\bf x}|{\\bf d})}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Optimisation is equivalent to maximum likelihood estimation\n", "\n", "




$$\\boldsymbol{v}_\\textrm{MLE}=\\underset{\\boldsymbol{v}\\in\\boldsymbol{V}}{\\textrm{argmin}}\\ln\\mathcal{L}({\\bf x}|\\boldsymbol{v},{\\bf d})$$
\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "### Optimisation with regularisation is equivalent to maximum a posteriori estimation\n", "\n", "$$\\boldsymbol{v}_\\textrm{MAP}=\\underset{\\boldsymbol{v}\\in\\boldsymbol{V}}{\\textrm{argmin}}\\ln\\mathcal{L}({\\bf x}|\\boldsymbol{v},{\\bf d})p(\\boldsymbol{v}|{\\bf d})$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "
\n", " \n", "### How do we know if we have the correct map?\n", "
\n", "
\n", " \n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Unfortunately you can't - but you can try and understand your output in terms of the network parameters" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# What can we say about the output given our network parameters?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# Describe our network parameters via distributions" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "There are several ways to do this\n", "\n", "- Stacked outputs\n", "- Variational Bayes (variational inference)\n", "- Bayesian neural networks (BNN)\n", "- BNN done effectively" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# Stacked outputs\n", "\n", "$$\\mathcal{P}({\\bf x}|\\boldsymbol{v},{\\bf d})\\approx\\textrm{KDE}(\\{{\\bf x}_n|n\\in\\textrm{number of NN}\\})$$\n", "\n", "
\n", "Train multiple models with different initialisations and use the distribution of the outputs\n", "\n", "- regression\n", " \n", "- classification\n", " \n", "- NDE\n", "
\n", "
\n", " \n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Unfortunately it doesn't really work unless you have LOADS of networks.\n", "\n", "(A bit dangerous - but less so than not doing it)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Variational Bayes (inference)\n", "
\n", "
\n", "

\n", " \n", "
\n", "
\n", "Describe each weight with a prior distribution (normally a Gaussian)\n", "\n", "$$p(\\boldsymbol{v}_i) = \\mathcal{N}(\\mu_i,\\sigma_i)$$\n", "\n", "Output of the network is also now a distribution\n", "\n", "$$\\mathscr{f}_\\boldsymbol{v}({\\bf d}) = \\mathcal{Q}({\\bf x}|\\boldsymbol{v},{\\bf d})$$\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Optimise the network using the KL-divergence (statistical difference measure (measure of the Shannon information)) between $\\mathcal{Q}({\\bf x}|\\boldsymbol{v},{\\bf d})$ and the true distribution $\\mathcal{P}({\\bf x}|\\boldsymbol{v},{\\bf d})$.\n", "\n", "- This moves the distributions of the weights around rather than the weights themselves\n", "\n", "- Really helps with optimisation and gives distributions as outputs!" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Very limiting on the form of the final distribution $\\mathcal{Q}({\\bf x}|\\boldsymbol{v},{\\bf d})$ which will never really become $\\mathcal{P}({\\bf x}|\\boldsymbol{v},{\\bf d})$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Bayesian neural networks\n", "\n", "Instead of forcing the form of the weights, we should really sample them\n", "\n", "$$\\mathcal{P}(\\boldsymbol{v}|{\\bf x},{\\bf d}) \\propto \\mathcal{L}({\\bf x}|\\boldsymbol{v},{\\bf d})p(\\boldsymbol{v}|{\\bf d})$$\n", "\n", "- Real outputs are available by marginalising the weights out\n", "\n", "- Many techniques for sampling, such as Markov chain Monte-Carlo" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "In practise the number of weights is too high dimensional to do this" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Hamiltonian Monte Carlo on network weights
(the proper way)\n", "
\n", "
\n", "Solve Hamilton's equations\n", "\n", "$$\\frac{\\partial\\boldsymbol{v}_i}{\\partial t}=\\frac{{\\bf p}_i}{m}\\textsf{ and }\\frac{\\partial{\\bf p}_i}{\\partial t}=-\\frac{\\partial\\ln\\mathcal{L}({\\bf x}|\\boldsymbol{v},{\\bf d})}{\\partial\\boldsymbol{v}_i}$$\n", "\n", "Samples from the weights $\\boldsymbol{v}$ and some (unimportant) velocity ${\\bf p}_i$ are found by solving the equations, conserving the likelihood.\n", "
\n", "
\n", " \n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### The problem is that you need the gradient of your likelihood with respect to your weights" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "# But that is exactly what you get with a neural network!" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "It is possible to sample the weights of your network in high dimensions using HMC!\n", "\n", "(it is still quite expensive)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ " " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Okay - now relax" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Machine learning is fun\n", "\n", "- It's super useful everywhere - especially in astrophysics\n", "\n", "- It's a statisticians best friend\n", "\n", "- You can do loads of cool things\n", "\n", "- You should never trust it\n", "\n", "- But you should also find ways to trust it because it's so useful" ] } ], "metadata": { "celltoolbar": "Diaporama", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" } }, "nbformat": 4, "nbformat_minor": 2 }